I hereby certify that this correspondence is being deposited with the U ja^Postal 
Service as Express Mail, Airbill No. EV323776881US, in an envelope^tlc^essedjto: 
Mail Stop Patent Application, Commissioner for Patents, P.O. Box/VfsoAlexap 
VA 22313-1450, on the date shown below. 



Dated: September 10, 2003 



Signature: 




Docket No.: 29926/39506 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
APPLICATION FOR UNITED STATES LETTERS PATENT 



Title: 



SUB-SAMPLING APPARATUS AND METHOD 
AND IMAGE SENSOR EMPLOYING THE SAME 



Wan-Hee Jo 



San 136-1, Ami-ri, Bubal-eub, Ichon-shi 
Kyoungki-do 467-860 



REPUBLIC OF KOREA 



PATENT 
29926/39506 



SUB-SAMPLING APPARATUS AND METHOD 
AND IMAGE SENSOR EMPLOYING THE SAME 

Technical Field 

An address sub-sampling technique is disclosed. More particularly, an address sub- 
5 sampling apparatus and method, and an image sensor employing the same are disclosed. 

Description of the Related Art 

Sub-sampling is also called sampling reduction or MUSE (Multiple sUb-nyquist 
Sampling Encoding) system. An image displayed on TV, for example, may be divided into 
two regions: a moving region and a still region. In the moving region, the image may not 

10 be sharp and there may also be blurring at a boundary part thereof. In addition, moving 

objects may have a low resolution. In view of the characteristics of the human eye, moving 
objects having a low resolution may not be sensed well by the human eye. Based on such 
characteristics, each image on TV in the MUSE system may be divided into a still region and 
a moving region, each of which uses different encoding techniques. The bandwidth of an 

15 original MUSE signal is 32 MHz where, in each of the above two regions, it is decreased by 
1/4 resulting in a bandwidth of 8 MHz. The MUSE signal is transmitted over a satellite 
transmission band of 27 MHz after FM modulating it. In other words, in the moving region, 
samples of the MUSE signal are compressed by 1/4 using sub-sampling and line 
misalignment sub-sampling at a horizontal axis. In the still region, samples of the MUSE 

20 signal are compressed by 1/2 by performing field misalignment sub-sampling, and field 
misalignment sub-sampling is performed once again at the time axis based on the concept 
that there is no picture change between adjacent frames. As a result, in both the moving 
region and the still region, pixels may be compressed by 1/4, and then the 1/4 compressed 
analog samples may be FM modulated to broadcast them over an artificial satellite channel 

25 of 27 MHz. 

Sub-sampling in an image sensor that is carried out during, for example, view finder 
or quick view operations may be used to increase rate of frame, while allowing the 
deterioration of resolution of a broader picture. For this sub-sampling operation, various 
modes may be provided based upon the design specification. In the pixel array structure, 
30 which provides a basic RGB (Red, Green, Blue) Bayer pattern, four pixels are composed as 
a basic unit. Sub-sampling is generally represented by m x n (herein, m and n are even 
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numbers), where m is the number of addresses to be outputted and n is the number of 
addresses to be skipped. 

For example, in the case of sub-sampling of 2 x 4, if it is assumed that the start 
address is "0", addresses to be outputted are in the order of 0, 1, 6, 7, 12, 13, ,. Thus, 2, 3, 4, 
5 5, 8, 9, 10, and 1 1 are addresses to be skipped. In such a case, when skipping from 1 to 6, 
the immediate next address would follow without any loss in view of time. However, the 
conventional method does not apply the skip function as described above, but masks output 
data in case addresses exist that need to be skipped. Thus, the conventional method has the 
shortcoming that it does not utilize the advantages of sub-sampling at the time axis. 

10 Summary 

In accordance with one aspect, an address sub-sampling apparatus includes a 
counting unit that generates a binary address of N bits, N being a natural number larger than 
2. The address sub-sampling apparatus also includes an address conversion unit that sub- 
samples the binary address of N bits to output a sub-sampled address having first, second 

1 5 and third bit groups. The sub-sampled address is arranged in order of the third, the first and 
the second bit groups from the MSB (Most Significant Bit). The first bit group, which is a 
combination of digits in the sub-sampled address corresponding to the number of addresses 
to be skipped, being set as "0", the second bit group, which includes the LSB (Least 
Significant Bit) corresponding to bits of the binary address, and the third bit group, which 

20 includes the MSB being set to shift the address subtracted by the number of bits in the first 
bit group from the MSB in the binary address. 

In accordance with another aspect, an image sensor of line scanning manner, includes 
a first counting unit that generates a first binary address of X bits in synchronization with a 
preset data clock signal, X being a natural number larger than 2. The image sensor also 

25 includes a first address conversion unit that sub-samples the first binary address of X bits to 
provide a first sub-sampled address having first, second and third bit groups, and a second 
counting unit that generates a second binary address of Y bits in synchronization with a 
preset line clock signal, Y being a natural number larger than 2. The image sensor further 
includes a second address conversion unit that sub-samples the second binary address of Y 

30 bits to output a second sub-sampled address having first, second and third bit groups, 

wherein each of the first and the second sub-sampled addresses is arranged in order of the 
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third, the first and the second bit groups from the MSB. The first bit group, which is a 
combination of digits in the sub-sampled address corresponding to the number of addresses 
to be skipped, being set as "0", the second bit group, which includes the LSB corresponding 
to bits of the binary address, and the third bit group, which includes the MSB being set to 
5 shift the address subtracted by the number of bits in the first bit group from the MSB in the 
binary address. 

In accordance with yet another aspect, an address sub-sampling method includes 
generating a binary address of N bits, N being a natural number larger than 2. The address 
sub-sampling method also includes sub-sampling the binary address of N bits to output a 

10 sub-sampled address that is provided in such a way that third, first, and second bit groups of 
the sub-sampled address are arranged in sequence from the MSB. The first bit group, 
which is a combination of digits in the sub-sampled address corresponding to the number of 
addresses to be skipped, being set as "0", the second bit group, which includes the LSB 
corresponding to bits of the binary address, and the third bit group, which includes the MSB 

15 being set to shift the address subtracted by the number of bits in the first bit group from the 
MSB in the binary address. 

In accordance with yet another aspect, a sub-sampling method for use in an image 
sensor includes generating a first binary address of X bits in synchronization with a preset 
data clock signal, and generating a second binary address of Y bits in synchronization with a 

20 preset line clock signal, both X and Y being natural numbers larger than 2. The sub- 
sampling method for use in an image sensor also includes sub-sampling the first and the 
second binary addresses to output first and second sub-sampled addresses that are provided 
in such a way that third, first, and second bit groups of each of the first and the second sub- 
sampled addresses are arranged in sequence from the MSB. The first bit group, which is a 

25 combination of digits in the sub-sampled address corresponding to the number of addresses 
to be skipped, being set as "0", the second bit group, which includes the LSB corresponding 
to bits of the binary address, and the third bit group, which includes the MSB being set to 
shift the address subtracted by the number of bits in the first bit group from the MSB in the 
binary address. 

30 A binary address 2 N of N bits provided from a counter is sub-sampled to output a 

sub-sampled address having first, second and third bit groups, wherein the sub-sampled 
address is arranged in order of the third, the first and the second bit groups from the MSB. 



3 



PATENT 
29926/39506 



The first bit group, which is a combination of digits in the sub-sampled address 
corresponding to the number of addresses to be skipped, being set as "0". The second bit 
group, which includes the LSB corresponds to bits of the binary address, and the third bit 
group, which includes the MSB is set to shift the address subtracted by the number of bits in 
5 the first bit group from the MSB in the binary address. 

The sub-sampling mode may be performed in real time by implementing an address 
conversion device using small-sized logic circuits that employ a bit shifting manner and 
inserts "0" into a specific bit. 

Brief Description of the Drawings 
10 Fig. 1 is an exemplary block diagram of an address sub-sampling apparatus in 

accordance with one embodiment; 

Figs. 2A to 2E are exemplary block diagrams illustrating one embodiment of a sub- 
sampling process; 

Figs. 3 A to 3C are tables showing the results of 10-bit sub-sampling; and 
15 Fig. 4 is an exemplary block diagram of an image sensor in accordance with another 

embodiment. 

Detailed Description 

Fig. 1 is an exemplary block diagram showing an address sub-sampling apparatus in 
accordance with one embodiment. The address sub-sampling apparatus includes a counter 
20 10 that generates a binary address (2° ~ 2 N_1 ) of N bits (herein, N is a natural number larger 
than 2), and an address conversion block 12 that outputs a sub-sampled address ([N-1 :0]) by 
sub-sampling the binary address (2° ~ 2 N_1 ). 

In addition, the address sub-sampling apparatus may further include a multiplexer 
(MUX) 1 1 that selects one of a plurality of predetermined sub-sampling modes in 
25 accordance with the binary address (2° ~ 2 N_1 ) of N bits. For purposes of illustration, the 
multiplexer 1 1 shown in Fig. 1 is provided in the address sub-sampling apparatus. Thus, 
the address conversion block 12 performs sub-sampling of the binary address in accordance 
with the address sub-sampling mode selected by the multiplexer 1 1 . 

As described above, the binary address of N bits includes 2° ~ 2 N "\ Therefore, if it 
30 is assumed that the sub-sampling processes are performed m x n times (herein, m is the 
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number of addresses to be outputted and n is the number of addresses to be skipped), m + n 
should be equal to or smaller than N- 1 , and m and n should both be even numbers. Thus, 

all possible modes are 0 x 0(1), 2 1 x2 1 ,2 1 x 2 2 , , 2 x 2 N -2 ! (2 N_1 ), 2 2 x mode (2 N " 2 ),. . .., 

2 N " 2 x mode (2 2 ), 2 N_1 x mode (2 1 ), and, therefore, the selectable mode number is 2 N -1, which 
5 is obtained by adding all of them through the use of a geometric series operation. 

The address conversion block 12 converts the binary address (2° ~ 2 N "*) into a sub- 
sampled address, which is formed in such a manner that third, first, and second bit groups of 
the binary address are arranged in sequence from the MSB (Most Significant Bit). That is, 
the first bit group, which is a combination of digits in the sub-sampled address 
10 corresponding to the number of addresses to be skipped, is set as "0", the second bit group, 
which includes the LSB (Least Significant Bit), is set as the bits of the binary address from 
the counter 10, and the third bit group, which includes the MSB, shifts the address subtracted 
by the number of bits in the first bit group from the MSB in the binary address from the 
counter 10. 

15 Details of the address sub-sampling process are given below. The following 

represents an algorithm of 10-bit addresses that is implemented by using known Verilog 
code: 



wire [N-1:0] ObyO = Counter [N-1:0]]; 
20 wire [N- 1 :0] 2By2 = {Counter [N-2 :2] } , 1 'b0, Counter [0] } ; 

Wire [N-1:0] 4by4 = {Counter [N-2:2]}, 1'bO, Counter [1:0]}; 

wire [N-1:0] 8by8 = {Counter [N-2:3]}, 1'bO, Counter [2:0]}; 

wire [N-1:0] 16byl6 = {Counter [N-2:4]}, 1'bO, Counter [3:0]}; 

wire [N-1:0] 2By6 = {Counter [N-3:l]}, 2'bOO, Counter [0]}; 
25 wire [N-1:0] 2Byl4 = {Counter [N-4:l]}, 3'b000, Counter[0]}; 

wire [N-l:0]2by30 = {Counter [N-5:l]}, 4'b0000, Counter[0]}; 



case(subsamp) 
'Zero Address=0by0; 
'One Address=2by2; 
'Two Address=4by4; 
'Three Address=8by8; 



//<=8 xl Mux 
110 x Sub-sampling mode 
112 x2 Sub-sampling mode 
1/4 x4 Sub-sampling mode 
IIS x 8 Sub-sampling mode 
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Tour Address=16byl6; //16 xl6 Sub-sampling mode 

'Five Address=2by6; 112 x6 Sub-sampling mode 

4 Six Address=2byl4; 112 xl4 Sub-sampling mode 

'Seven Address=2by30; 112 x30 Sub-sampling mode 

5 

The above algorithm is represented in Verilog code for performing the sub-sampling 
operation, and is implemented to synthesize digital logic circuits. Specifically, the address 
conversion block 12 receives the binary address of N bits outputted from the counter 10 and 
performs bit operations to generate each sub-sampled address, thereby providing 

10 corresponding sub-sampled addresses in accordance with eight (Zero to Seven) 
predetermined sub-sampling modes as described above. 

As is known in the art, Verilog code is classified by wire sentence and case sentence, 
wherein the wire sentence performs bit operations on the outputs from the counter 10. For 
example, in the case of sub-sampling of 2By2 (2 x 2) mode(l), 2 x 2 of wire sentence is set 

15 in such a way that the upper bits [N-2:l] of the output from the counter 10 correspond to bits 
[N-l : 1] of 2By2. In other words, the remaining address, which is subtracted by the number 
of bits in the first bit group to be set as "0", is shifted from the MSB in the output of the 
counter 10. 

The first bit group, which consists of a combination (2 1 ) of digits in the address to be 
20 sub-sampled, which is equal to the number (2) of addresses to be skipped, bit [1] of 2By2, is 
set as "0"(1 'b0). The lower bit[0] of the output from the counter 10 corresponds to "0" of 
2By2 (Counter[0]). That is, the second bit group including the LSB is set as the bit applied 
to the counter 10. 

In the case of sub-sampling of 2By30, 2By30 of the wire sentence sets in such a way 
25 that upper bits [N-5:l] of the output from the counter 10 correspond to bit [N-l :5] of 2By30, 
bits [4:1] of 2By30 are fixed to "0", and lower bit [0] of the counter 10 corresponds to bit [0] 
of 2By2. One of the mode signals from ObyO to 2By30 obtained by doing so is selectively 
output from the multiplexer 11, and depends on sub-sampling given by the case sentence. 
Although an embodiment using the Verilog language is illustratively given for the sake of 
30 simplicity, other languages such as VHDL (Very-High speed integrated Description 
Language), etc. may be used. 

Figs. 2A to 2E are exemplary block diagrams illustrating one embodiment of a sub- 
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sampling process. In this embodiment, a process of sub-sampling a binary address of 10 
bits is provided through several predetermined sub-sampling modes. Referring to Fig. 2A, 
address [9:0], which is output from the 10-bit counter 10, is sub-sampled by the address 
conversion block 12 set in the sub-sampling mode of 2 x 2 shown in Fig. 2B. Here, the first 
5 bit group <1>, which is a combination of digits in the address to be sub-sampled which is 
equal to the number "2" of the address to be skipped, is set as "0". The second bit group 
<2>, which includes the LSB and is a lower bit of the first bit group <1>, is set as bit [0] sent 
from the counter 10. In the third bit group <3>, which consists of upper bits of the first bit 
group <1> and includes the MSB, address [8:1] subtracted by the number (1) of bits in the 
10 first bit group <1> from the MSB in the address from the counter 10 is shifted to the upper 
bits. 

Referring to Fig. 2C, address [9:0], which is output from the 10-bit counter 10 of Fig. 
2 A is sub-sampled by the address conversion block 12, which is set in the sub-sampling 
mode of case (2 x 14). Here, the first bit group <1>, which is a combination of digits in the 

15 , address to be sub-sampled which is equal to the number "14" of the address to be skipped, is 
set as "0". The second bit group <2>, which includes the LSB and is a lower bit of the first 
bit group <1>, is set as bit [0] from the counter 10. In the third bit group <3>, which 
consists of upper bits of the first bit group <1> and includes the MSB, address [6:1] 
subtracted by the number (3) of bits in the first bit group <1> from the MSB in the address 

20 from the counter 1 0 is shifted to the upper bits. 

Referring to Fig. 2D, address [9:0], which is output from the 10-bit counter 10 of Fig. 
2 A, is sub-sampled by the address conversion block 12 set in the sub-sampling mode of 2 x 
30. Here, the first bit group <1>, which is a combination of digits in the address to be sub- 
sampled which is equal to the number "30" of the address to be skipped is set as "0". The 

25 second bit group <2>, which includes the LSB and is a lower bit of the first bit group <1>, is 
set as bit [0] from the counter 10. In the third bit group <3>, which consists of upper bits of 
the first bit group <1> and includes the MSB, address [5:1] subtracted by the number (4) of 
bits in the first bit group <1> from the MSB in the address from the counter 10 is shifted to 
the upper bits. 

30 Referring to Fig. 2E, address [9:0], which is output from the 10-bit counter 10 of Fig. 

2 A, is sub-sampled by the address conversion block 12 set in the sub-sampling mode of 4 x 
4. Here, the first bit group <1>, which is a combination of digits in the address to be sub- 
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sampled which is equal to the number "4" of the address to be skipped, is set as "0". The 
second bit group <2>, which includes the LSB and is a lower bit of the first bit group <1>, is 
set as bit [1:0] from the counter 10. In the third bit group <3>, which consists of the upper 
bits of the first bit group <1> and includes the MSB, address [8:1] subtracted by the number 
5 (1) of bits in the first bit group <1> from the MSB in the address from the counter 10 is 
shifted to the upper bits. 

Figs. 3 A to 3C show sub-sampling tables representing the sub-sampling results of 10 
bits in accordance with one embodiment. In other words, Figs. 3A to 3C represent a sub- 
sampled address that is obtained by sub-sampling the binary address from the counter 10 in 

10 synchronization with the clock signal (Clk) from the counter 10 at the address conversion 
block 12. The output of the counter 10 is represented by the decimal number on the left 
part of the table, and the decimal number and binary number of its address are represented 
together on the upper side of the table. Further, it can be seen that the table depends on 
rules of the first to third bit groups <1> ~ <3> of the sub-sampled address. That is, it can be 

15 seen that columns of the first bit group <1> represented by white in each mode are all set as 
"0". 

Fig. 4 is a block diagram illustrating an image sensor in accordance with another 
embodiment. The image sensor includes a first counter 40a that generates a binary address 
of X (herein, X is a natural number larger than 2) bits in synchronization with a preset data 

20 clock signal (Clled) in a line scan manner that scans one line, i.e., column in synchronization 
with a line clock signal and then transmits it at once. The image sensor also includes a first 
address conversion block 42a that sub-samples the binary address (2° ~ 2 X_1 ) from the first 
counter 40a to provide a sub-sampled address [X-1:0], i.e., column address. 

The image sensor further includes a second counter 40b, which is operated in 

25 synchronization with a line clock signal (ClkJ), that generates a binary address (2° ~ 2 Y ' 1 ) of 
Y (herein, Y is a natural number larger than 2) bits, and a second address conversion block 
42b that sub-samples the binary address (2° ~ 2 YA ) from the second counter 40b to provide a 
sub-sampled address [Y-1:0], i.e., row address. 

As illustrated in Fig. 4, the image sensor further includes a first multiplexer 41a that 

30 selects one of a plurality of predetermined sub-sampling modes capable of outputting from 
the binary address (2° - 2 X ~ 1 ) of X bits, and a second multiplexer 41b for selecting one of a 
plurality of predetermined sub-sampling modes capable of outputting from the binary 
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address (2° ~ 2 Y1 ) of Y bits. Thus, each of the address conversion blocks 42a and 42b 
performs a sub-sampling operation based on the corresponding mode selected by each of the 
multiplexers 41a and 41b. Here, a binary address of N bits includes 2° ~ 2 X_1 for column 
and 2° ~ 2 Y ' 1 for row and, therefore, if it is assumed that the sub-sampling processes are 
5 performed m x n times (herein, m is the number of addresses to be outputted and n is the 

number of addresses to be skipped), m + n should be smaller than or equal to N-l, and m and 
n should both be even numbers. Thus, it can be seen that the number of selectable modes 
for column and row become 2° ~ 2 X_1 and 2° - 2 Y_1 , respectively, by performing a geometric 
series operation. 

10 The first and the second address conversion blocks 40a and 40b convert the binary 

addresses of 2° ~ 2 X " 1 and 2° ~ 2 Y " 1 into sub-sampled addresses, column address and row 
address, which are made in such a way that the third, the first, and the second bit groups are 
arranged in sequence from the MSB thereof. The first bit group, which is a combination of 
digits in the sub-sampled address corresponding to the number of addresses to be skipped, is 

15 set as "0", the second bit group corresponds to bits of the binary address from the counter, 
and in the third bit group including the MSB, the address subtracted by the number of bits in 
the first bit group from the MSB in the address from the counter is shifted to upper bits. 

In general, since the image sensor supports operation of line scan manner, as 
mentioned above, the first counter 40a, the first multiplexer 41a, and the first address 

20 conversion block 42a generate the column address, and the second counter 40b, the second 
multiplexer 41b, and the second address conversion block 42b produce the row address. 
The first counter 40a, which is a circuit for generating the column address [X-l :0] is 
operated based on the rate of output of the data, i.e., in synchronization with the data clock 
signal (Clk-d), and the second counter 40b, which is a circuit for generating the row address 

25 [Y-l :0] is operated based on the rate of output of the data, i.e., in synchronization with the 
line clock signal (Clk.1). 

The line clock signal (Clk.1) and the data clock signal (Clk.d) have the relationship of 
Eq. (1) as follows: 

30 Clk.1 = tbiank + (Wc x Clk.d) Eq.(l), 

wherein "tbiank " which is the time difference between lines, represents a time interval 
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from completion of scanning of a line to before issuing of line clock signal required for 
scanning of a next line. In addition, the time "t b i an k" includes a time taken for CDS 
(Correlated Double Sampling), etc. 

Specifically, referring to Eq. (1), the line clock signal (Clk.l) has a period that is 
calculated by adding the time (t b iank) between the lines to a value that is obtained by 
multiplying the width of the column of a pixel arrangement block by the data clock signal 
(Clk-d). For instance, if it is assumed that VGA (Video Graphics Array) is capable of 
representing 256 colors and its resolution is 640 x 480, the period of the line clock signal 
(Cllel) is longer than that of the data clock signal (Clk_d) since the line clock signal is 
obtained by adding an additional time to a value that is derived by multiplying the period of 
the data clock by 640. Further, the fist address conversion block 42a and the second 
address conversion block 42b employ circuits having the same concept except that the 
number of bits of addresses is different from each other. 

The sub-sampling method of the image sensor will be omitted here for the purpose of 
simplicity because it may be implemented in the same manner as the inventive sub-sampling 
described above. As a result, since the address conversion blocks are designed using a 
small-sized digital logic circuit and may perform the sub-sampling mode in real time, users 
have access to a wide range of products and may easily support the back-end processing 
such as, for example, image signal process, image compression, software process, etc. 

While the present disclosure has been described with respect to the particular 
embodiments, it will be apparent to those skilled in the art that various changes and 
modifications may be made without departing from the spirit and scope of the disclosure as 
defined in the following claims. 
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